Some Explanatory Factors for Software Quality

Even in a world of agile methods and Internet-time processes, quality is a crucial attribute of software. A number of explanatory factors for software quality have been proposed, but the empirical evidence on the effect of these factors has been mixed. One reason for the mixed evidence is that surrogates are frequently used that capture the desired factor poorly. For example, it is widely agreed that the competence of the people doing the work is fundamental to the quality of the work done. How does one measure competence? Even if we agree that surrogates such as years of experience are inadequate, better measures may not be readily available. After reviewing the evidence supporting various proposed factors, data from the Personal Software Process (PSP) is analyzed to see what the impact of some of those factors is on software quality as measured by defects found in testing. The factors considered include process factors, such as design time, and non-process factors, such as programming language. Arguably the most important factor to be considered is the competence of the people doing the work, and a number of surrogates are considered. One of the greatest challenges in empirical software engineering is the variability associated with individual differences, and the PSP data shows that, even though performance improves and variability decreases as disciplined processes are instilled, the dominant factor in superior performance remains the competence of the individual professional. The paper closes with a discussion of the issues associated with generalizing from PSP data to an industry environment, since PSP assignments are “toy problems” compared to the complex systems commonly demanded today. . Published as M.C. Paulk, “Some Explanatory Factors for Software Quality,” Third International Conference on Software Quality, Dallas, October 2003.